1. Field of the Invention
The present invention relates to an image retrieving technique for judging whether an image in an area of an input image is similar to or the same as a predetermined reference image or not.
2. Description of the Related Art
Color histogram is conventionally used for judging whether an image in an area of an input image is similar to or the same as a predetermined reference image or not. In the method using the color histogram, a color histogram of the reference image and a color histogram of an image in a predetermined area in the input image are compared. The area in the input image to be compared is moved at a predetermined pitch in the horizontal and vertical directions on the whole input image. The identity or the similarity of the color histogram of each portion of the input image and the color histogram of the reference image are calculated. The area in the input image having the identity or the largest similarity is judged as the area the same as or similar to the reference image. The size of the area in the input image to be compared can be varied corresponding to the size of the reference image.
For increasing the processing speed of the image retrieving by the method using the color histogram, it is proposed to vary the pitch of the movement of the area to be compared corresponding to the similarity of the color histograms (see collection of congress of electronic information and communication D-II Vol.J81-D-II No.9 pp.2035-2042 September 1998). In this modification, when the area to be compared is in the vicinity of the area having the lower similarity, the pitch of the movement of the area is varied to be larger. When the area to be compared is in the vicinity of the area having the higher similarity, the pitch of the movement of the area is varied to be smaller. As a result, the processing speed of the image reference can be made faster.
In the above-mentioned conventional methods, the color histogram is calculated with respect to each image in the area and compared with that of the reference image. Furthermore, the size of the area to be compared can be varied corresponding to the size of the reference image, so that the burden of the image processing becomes larger. Thus, a high performance computer is necessary for processing the image reference. Furthermore, when the color histogram having a fine resolution of gradation is used, the quantity of the calculation necessary for referencing the color histograms becomes much larger.
Actually, it is desired to know whether a predetermined kind of image such as a person is included in the input image or not, instead of judging whether the same image as the reference image is included in the input image or not. In such the case, the size of the area to be compared is generally known. Thus, it is desired to propose a new method for judging whether a predetermined kind of image is included in the input image or not by a calculation performance such as a one-chip microcomputer used in a household electric appliance.
On the other hand, in the image retrieving of the input image by using the color histogram, the number of image data of the input image or the reference image is sometimes small, when the density of the image is small or when the size of the image to be compared is small. In such the case, the color histogram will take a comb shape or a discrete histogram including the gradation of zero degree.
An example that both of the numbers of the image data of the input image and the reference image are small is described with reference to FIGS. 30A to 30E. FIG. 30A shows an input image 201. FIG. 30B shows a reference image 202. Numeral 203 in FIG. 30A designates an area to be retrieved. FIG. 30C shows a normalized color histogram of the area 203. FIG. 30D shows a normalized color histogram of the reference image 202. Hereupon, in the normalized color histogram, a value that the number of the pixels having the same gradation divided by the number of the total pixels is used as the degree of each gradation, and the sum the degrees of every gradations is normalized to be “1”.
When the numbers of the image data of the input image 201 and the reference image 202 are small, the color histograms of them will be the discrete comb shape including the gradation of zero degree, as shown in FIGS. 30C and 30D. Furthermore, when the luminance in the input image 201 and/or the reference image 202 are/is varied or when the blushing occurs in one or both of the images, the color histogram of the input image 201 will be discrepant from that of the reference image 202, as shown in FIG. 30E, so that the similarity between the input image 201 and the reference image 202 becomes much lower. Thus, an area to be retrieved will erroneously be judged as the area not including the reference image. The similarity is a value calculated that the number of degrees in the color histograms of the input image 201 and the reference image 202 are compared with respect to each gradation, and the smaller degrees are added with respect to every gradations.
Another example that the number of the image data of the input image 211 is largely different from that of the reference image 212 is described with reference to FIGS. 31A to 31G. FIG. 31A shows an input image 211. FIG. 31B shows a reference image 212. Numeral 213 in FIG. 31A designates an area to be retrieved, and numeral 214 designates another area not to be retrieved. FIG. 31C shows a normalized color histogram 215 of the area 213. FIG. 31D shows a normalized color histogram 216 of the area 214. FIG. 31E shows a normalized color histogram 217 of the reference image 212.
In this example, the number of the image data of the area 213 is smaller than that of the area 214, but the number of the image data of the reference image 212 is similar to that of the area 214.
As can be seen from FIG. 31C, the color histogram 215 which is formed by basing the small number of the image data has a discrete comb shape including the gradation of zero degree. On the other hand, as can be seen from FIGS. 31D and 30E, the color histograms 216 and 217 which are formed by basing the relatively large number of the image data respectively have successive curves taking positive values.
FIG. 31F shows the color histograms 215 and 217 which are superimposed on the same coordinates. In FIG. 31F, hatched portions 218 correspond to the similarity of the color histogram 215 of the area 213 and the color histogram 217 of the reference image 212. FIG. 31G shows the color histograms 216 and 217 which are superimposed on the same coordinates. In FIG. 31G, a hatched portion 219 corresponds to the similarity of the color histogram 216 of the area 214 and the color histogram 217 of the reference image 212.
As can be seen from FIGS. 31F and 31G, the color histogram 215 of the area 213 has the comb shape, so that the similarity of the hatched portions 218 is smaller than that of the hatched portion 219 with respect to the color histogram 217. Thus, the area 214 which is not to be retrieved will erroneously be retrieved as the area including the reference image instead of the area 213 to be retrieved.
As mentioned above, when the color histogram becomes the comb shape, the similarity of the histogram of an area of the input image with respect to that of the reference image becomes lower even though the gradation is discrepant a little. Especially, when the luminance in the input image is varied, the image retrieving performance will become much lower. Furthermore, when the number of the image data of the input image or the reference image is largely different from the number of the image data of the area to be compared, the image retrieving performance will be reduced.